Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planning

Goal-Conditioned Reinforcement Learning (GCRL) can enable agents to spontaneously set diverse goals to learn a set of skills. Despite the excellent works proposed in various fields, reaching distant goals in temporally extended tasks remains a challenge for GCRL. Current works tackled this problem by leveraging planning algorithms to plan intermediate subgoals to augment GCRL. Their methods need two crucial requirements: (i) a state representation space to search valid subgoals, and (ii) a distance function to measure the reachability of subgoals. However, they struggle to scale to high-dimensional state space due to their non-compact representations. Moreover, they cannot collect high-quality training data through standard GC policies, which results in an inaccurate distance function. Both affect the efficiency and performance of planning and policy learning. In the paper, we propose a goal-conditioned RL algorithm combined with Disentanglement-based Reachability Planning (REPlan) to solve temporally extended tasks. In REPlan, a Disentangled Representation Module (DRM) is proposed to learn compact representations which disentangle robot poses and object positions from high-dimensional observations in a self-supervised manner. A simple REachability discrimination Module (REM) is also designed to determine the temporal distance of subgoals. Moreover, REM computes intrinsic bonuses to encourage the collection of novel states for training. We evaluate our REPlan in three vision-based simulation tasks and one real-world task. The experiments demonstrate that our REPlan significantly outperforms the prior state-of-the-art methods in solving temporally extended tasks.Comment: Accepted by 2023 RAL with ICR

Diverse Knowledge Distillation for End-to-End Person Search

Person search aims to localize and identify a specific person from a gallery of images. Recent methods can be categorized into two groups, i.e., two-step and end-to-end approaches. The former views person search as two independent tasks and achieves dominant results using separately trained person detection and re-identification (Re-ID) models. The latter performs person search in an end-to-end fashion. Although the end-to-end approaches yield higher inference efficiency, they largely lag behind those two-step counterparts in terms of accuracy. In this paper, we argue that the gap between the two kinds of methods is mainly caused by the Re-ID sub-networks of end-to-end methods. To this end, we propose a simple yet strong end-to-end network with diverse knowledge distillation to break the bottleneck. We also design a spatial-invariant augmentation to assist model to be invariant to inaccurate detection results. Experimental results on the CUHK-SYSU and PRW datasets demonstrate the superiority of our method against existing approaches -- it achieves on par accuracy with state-of-the-art two-step methods while maintaining high efficiency due to the single joint model. Code is available at: https://git.io/DKD-PersonSearch.Comment: Accepted to AAAI, 2021. Code is available at: https://git.io/DKD-PersonSearc

Intelligent ZHENG Classification of Hypertension Depending on ML-kNN and Information Fusion

Hypertension is one of the major causes of heart cerebrovascular diseases. With a good accumulation of hypertension clinical data on hand, research on hypertension's ZHENG differentiation is an important and attractive topic, as Traditional Chinese Medicine (TCM) lies primarily in “treatment based on ZHENG differentiation.” From the view of data mining, ZHENG differentiation is modeled as a classification problem. In this paper, ML-kNN—a multilabel learning model—is used as the classification model for hypertension. Feature-level information fusion is also used for further utilization of all information. Experiment results show that ML-kNN can model the hypertension's ZHENG differentiation well. Information fusion helps improve models' performance

The correlation of the intestinal with pharyngeal microbiota in early neonates

IntroductionThe gut-lung axis has long been recognized as an important mechanism affecting intestinal and lung immunity. Still, few studies have examined the correlation between the intestinal and pharyngeal microbiota in early neonates, especially when feeding patterns are one of the main drivers of microbiota development.MethodsTo explore the composition and function of intestinal and pharyngeal microbiota and to analyze the effect of limited formula feeding on the initial microbiota colonization in early full-term neonates, we characterized the stool and oropharyngeal microbiota of 20 healthy full-term newborns sampled on days 0 and 5–7 after birth using 16S rRNA gene sequencing. Based on the sequencing results, a comparison was made of the compositions and functions of the intestinal and oropharyngeal microbiota for analysis.Results and discussionAt the phylum level, Firmicutes, Actinobacteria, Proteobacteria, and Bacteroidetes were the most abundant in both niches. At the genus level, the species of pioneer bacteria were rich in the intestine and oropharynx but low in abundance on day 0. On days 5–7, Bifidobacterium (25.40%) and Escherichia-Shigella (22.16%) were dominant in the intestine, while Streptococcus (38.40%) and Staphylococcus (23.13%) were dominant in the oropharynx. There were eight core bacteria genera in the intestine and oropharynx on days 5–7, which were Bifidobacterium, Escherichia-Shigella, Staphylococcus, Streptococcus, Bacteroides, Parabacteroides, Rothia, and Acinetobacter. As indicated by PICRUSt analysis, on days 5–7, the intestinal microbiota was more predictive than the oropharyngeal microbiota in transcription, metabolism, cell motility, cellular processes and signaling, and organismal system function in the KEGG pathway. Compared to exclusive breastfeeding, limited formula feeding (40–60%) had no significant effect on the neonatal intestinal and oropharyngeal microbiota composition during the initial colonization period. Our results suggest that the initial colonization of microbiota is closely related to the ecological niche environment in the intestine and oropharynx, with their core microbiota being closely correlated. We found that early limited formula feeding could not significantly affect the initial colonization of microbiota in the intestine and oropharynx